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Abstract 

The  verification  of  a  local  minimizer  of  a  general  (i.e.,  nonconvex)  quadratic 
program  is  in  general  an  NP-hard  problem.  The  difficulty  concerns  the  op¬ 
timality  of  certain  points  (which  we  call  dead  points)  at  which  the  first-order 
necessary  conditions  for  optimality  are  satisfied,  but  strict  complementarity 
does  not  hold. 

One  important  class  of  methods  for  solving  general  quadratic  programming 
problems  are  called  inertia-controlling  quadratic  programming  (1CQP)  meth¬ 
ods.  We  derive  a  computational  scheme  for  proceeding  at  a  dead  point  that  is 
appropriate  for  a  general  ICQP  method. 

Keywords:  Quadratic  programming,  local  minimizer,  NP-hardness. 


1.  Introduction 

The  general  quadratic  programming  problem  is  to  find  a  local  minimizer  of  a  quad¬ 
ratic  function  subject  to  linear  constraints.  In  this  paper,  the  problem  considered 
is  of  the  form 

minimize  ¥>(x)  =  cTx  +  \xTfIx 

^  2  (1.1) 

subject  to  Ax  >  0, 
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where  the  Hessian  matrix  II  is  symmetric,  and  A  is  an  mi,  X  n  matrix.  Of  particular 
interest  is  the  nonconvex  case  where  II  has  an  arbitrary  distribution  of  positive, 
negative  and  zero  eigenvalues. 

Our  attention  will  focus  on  the  class  of  inertia-controlling  methods  for  general 
quadratic  programming.  Inertia-controlling  quadratic  programming  (ICQP)  meth¬ 
ods  use  a  linearly  independent  subset  of  the  constraints  known  as  the  working  set 
to  define  a  search  direction  and  multiplier  estimates.  A  unique  feature  of  ICQP 
methods  is  that  constraint  deletions  are  restricted  so  as  to  control  the  inertia  of 
the  reduced  Hessian ,  which  is  never  permitted  to  have  more  than  one  nonpositive 
eigenvalue.  Fletcher  [FleTl]  proposed  the  first  ICQP  method,  and  various  methods 
within  this  class  have  also  been  proposed,  see  for  example  Gill  ct  al.  [GMSW84]  and 
Gould  [Gou86]. 

For  any  nonconvex  quadratic  program  there  may  exist  certain  dead  points  at 
which  all  quadratic  programming  methods  will  find  if  difficult  to  proceed  (see  Sec¬ 
tion  2.6,  for  a  precise  definition  of  a  dead  point).  The  difficulty  arises  because  the 
verification  of  such  a  point  as  a  local  minimizer  of  (1.1)  is  an  NP-hard  problem  — 
see  Murthy  and  Kabacli  [MK87]  and  Pardalos  and  Schnitger  [PS88].  Unfortunately, 
even  if  lower  values  of  9  do  exist  in  the  neighborhood  of  a  dead  point,  any  number  of 
constraints  may  need  to  be  deleted  simultaneously  in  order  to  compute  a  direction 
of  improvement.  Since  existing  ICQP  methods  can  delete  only  one  constraint  at  a 
time,  they  may  be  unable  to  proceed  from  a  dead  point. 

The  difficulties  associated  with  existing  ICQP  methods  at  a  dead  point  may  be 
contrasted  with  the  difficulties  associated  with  the  simplex  method  at  a  degenerate 
vertex.  The  simplex  method  is  able  to  keep  iterating  at  a  degenerate  vertex,  but  it 
may  be  necessary  to  perform  a  (possibly  infinite)  number  of  iterations  during  which 
the  working  set  changes,  but  the  vertex  remains  the  same.  By  contrast,  existing 
ICQP  methods  may  terminate  prematurely  at  a  dead  point  which  is  not  a  local 
minimizer. 

If  progress  is  to  be  made  at  a  dead  point,  a  scheme  must  be  devised  for  the  iden¬ 
tification  and  simultaneous  deletion  of  more  than  one  aint  from  the  working 

set.  The  computational  and  theoretical  properties  of  ,':'  h  >  scheme  are  presented 
in  this  paper.  In  Section  G  we  show  that  at  a  dead  pc.  the  proposed  method 
behaves  in  a  similar  wav  to  the  simplex  at  a  degenerate  vertex — i.e.,  the  algorithm 
is  able  to  proceed,  but  there  exists  the  danger  of  cycling.  In  order  to  guarantee 
finite  termination,  it  is  necessary  to  introduce  procedures  (such  as  a  lexicographic 
ordering)  similar  to  the  anti-cycling  procedures  used  in  the  simplex  method. 

In  an  ICQP  method,  the  reduced  Hessian  is  required  to  be  positive  definite  at  the 
starting  point.  In  order  to  ensure  this,  it  might  be  necessary  to  introduce  artificial 
constraints.  (The  importance  of  such  constraints  is  explained  in  Section  2.5.)  Un- 
Srtnnatelv,  the  presence  of  these  constraints  may  introduce  additional  dead  points 
which  are  not  present  in  the  original  problem.  However,  we  show  that  the  com¬ 
putational  scheme  derived  is  able  to  treat,  the  artificial-constraint  case  so  that  the 
artificial  constraints  cause  no  additional  difficulties. 

In  order  to  describe  the  new  scheme,  we  first  review  results  on  necessary  and 
sufficient  conditions  for  optimality  in  general  quadratic  program  mine.  For  a  discus- 
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A4.  The  point  x  satisfies  the  first-order  necessary  conditions  for  optimality,  i.e., 
there  exists  a  nonnegative  Lagrange  multiplier  vector  p  such  that  x  and  p 
satisfy  the  Karush-Kuhn-Tucker  equations 


I  H  At 
\  A  0 


(2.1) 


A5.  The  reduced  Hessian,  ZTH  Z,  is  positive  definite. 


2.4.  The  inertia  of  a  matrix 


Let  M  be  any  symmetric  matrix.  We  denote  by  ip(M),  in{M)  and  iz(M)  respectively 
the  (nonnegative)  numbers  of  positive,  negative  and  zero  eigenvalues  of  M.  The 
inertia  of  M — denoted  by  In(M) — is  the  associated  integer  triple  [ip,in,iz).  The 
following  lemma  states  an  important  relationship  between  the  inertia  of  the  KK T - 
matrix 


K  = 


H 

A 


AT 

0 


and  the  reduced  Hessian. 


Lemma  2.1.  Given  assumptions  A3  and  A5,  the  inertia  of  the  KKT  matrix  K  is 
(n,m,0). 


Proof.  See  Gould  [Gou85,  Lemma  3.4].  | 


Lemma  2.1  implies  that  Ii  is  nonsingular,  so  that  the  Lagrange  multipliers  in 
(2.1 )  are  unique. 


2.5.  Inertia-controlling  methods  for  quadratic  programming 

Associated  with  each  iteration  of  an  ICQP  method  is  a  linearly  independent  subset 
of  the  constraints  known  as  the  working  set.  The  working  set  at  the  initial  point  xg 
must  be  chosen  so  that  the  reduced  Hessian  is  positive  definite.  Thereafter,  working 
set  changes  by  only  one  constraint  at  each  iteration  and  the  reduced  Hessian  is  never 
permitted  to  have  more  than  one  nonpositive  eigenvalue. 

ICQP  methods  depend  critically  on  a  procedure  for  finding  an  initial  working 
set  with  an  associated  positive-definite  reduced  Hessian.  In  order  to  ensure  that 
the  reduced  Hessian  is  positive  definite,  the  initial  working  set  may  need  to  include 
"artificial”  constraints  that  are  not  specified  in  the  original  problem.  The  only  re¬ 
quirement  for  an  artificial  constraint  is  linear  independence  from  constraints  already 
in  the  working  set.  Artificial  constraints  do  not  restrict  the  feasible  region,  since  the 
direction  of  the  inequality  need  not  be  defined.  As  soon  as  an  artificial  constraint 
can  be  removed  from  the  working  set,  it  is  eliminated  from  the  problem.  The  strat¬ 
egy  for  choosing  artificial  constraints  depends  on  the  mechanics  of  the  particular  QP 
method  and  procedures  for  finding  the  initial  working  set  are  usually  dependent  on 
the  method  used  to  solve  the  KKT  system.  A  simple  example  of  a  problem  requiring 
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■j 


artificial  constraints  is  given  in  Section  5.  We  emphasize  that  artificial  constraints 
are  not  part  of  the  original  problem,  but  are  an  artefact  of  the  solution  method. 
1  lie  original  constraints  of  the  problem  are  referred  io  as  regular  constraints. 

Once  a  constrained  minimizer  is  found,  an  ICQP  algorithm  proceeds  by  deleting 
one  constraint  from  the  working  set  and  finding  either  a  feasible  descent  direction 
or  a  feasible  direction  of  negative  curvature.  The  constraint  deletion  is  permitted 
only  if  the  reduced  Hessian  is  positive  definite. 

All  ICQP  methods  generate  search  vectors  and  multipliers  that  satisfy  the  KKT 
equations.  However,  the  equations  may  be  solved  either  implicitly  or  explicitly,  in 
which  case  one  iCQF  algorithm  may  appear  to  be  v.ry  different  from  another.  In 
this  paper,  only  the  form  of  equations  to  be  solved  is  stated.  Tor  a  discussion  on 
the  relationship  between  different  ICQP  methods,  see  Gill  et  al.  [GMSW88]. 

Dead  points  with  only  regular  constraints  will  be  treated  first.  In  Section  5,  we 
consider  the  case  when  artificial  constraints  are  present. 


2.6.  Dead  points 

if  .4  contains  only  regular  constraints,  a  dead  point  is  defined  to  be  a  point  satisfying 
assumptions  A4  and  A5  for  which  one  or  more  components  of  the  Lagrange  multiplier 
vector  p  are  zero.  We  emphasize  that  such  a  point  may  not  be  a  local  minimizer. 

Since  a  dead  point  satisfies  the  first-order  necessary  conditions  for  optimality, 
there  exists  no  feasible  descent  direction.  Therefore,  it  is  necessary  to  find  a  feasible 
direction  of  negative  curvature  if  an  ICQP  method  is  to  proceed  to  find  a  local 
minimizer.  Unfortunately,  it  may  be  impossible  to  compute  a  feasible  direction  of 
negative  curvature  by  deleting  only  one  constraint  at  a  time,  as  can  be  seen  from 
the  following  problem. 


minimize  -xixo 
r€*2 

subject  to  0  <  Xj  <  1  (2.2) 

0  <  x2  <  1. 

If  the  starting  point  is  the  origin,  and  both  active  constraints  are  in  the  working 
M't .  assumptions  A1-A5  are  satisfied.  However,  if  either  of  the  constraints  is  deleted 
from  the  working  set,  the  resulting  reduced  Hessian  is  positive  semidefinitc  and 
singular.  Therefore,  no  feasible  direction  of  negative  curvature  may  be  computed  by 
deleting  only  one  constraint.  No  more  than  one  constraint  may  be  deleted  from  the 
working  set,  since  constraint  deletion  is  permitted  only  when  the  reduced  Hessian  is 
positive  definite.  Therefore,  an  ICQP  method  must  terminate  at  this  point,  although 
the  origin  is  not  a  local  minimizer  for  the  QP. 

In  this  situation — where  neither  a  feasible  descent  direction  nor  a  feasible  di¬ 
rection  of  negative  curvature  may  be  found  by  deleting  only  one  constraint — it  is 
necessary  to  develop  a  scheme  for  deleting  more  than  one  constraint  simultaneously 
if  an  ICQP  method  is  to  proceed. 
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2.7.  Optimality  conditions 


In  this  section,  necessary  and  sufficient  conditions  for  x  to  be  a  local  minimizer 
under  assumptions  A1-A5  are  reviewed.  It  will  be  useful  to  distinguish  between 
constraints  with  positive  and  zero  multipliers.  Without  loss  of  generality  we  may 
assume  that  the  rows  of  A  are  ordered  such  that 


where  A+  corresponds  to  rows  with  positive  Lagrange  multipliers  and  A0  corresponds 
to  rows  with  zero  Lagrange  multipliers.  Let  m+  denote  the  number  of  rows  in 
/4+,  and  let  m0  denote  the  number  of  rows  in  .40.  Also,  let  p  +  denote  the  vector 
containing  the  m+  positive  components  of  f. i . 

The  following  two  necessary  and  sufficient  conditions  for  x  being  a  local  mini¬ 
mizer  for  (1.1)  when  assumption  A2  holds  are  given  by  Majthay  [MajTl]  and  Con- 
tesse  [Con80]. 


Cl.  The  point  x  satisfies  the  first-order  necessary  conditions  for  optimality,  i.e., 
there  exists  a  nonnegative  Lagrange  multiplier  vector  p,  such  that  x  and  p 
satisfy  the  KKT  equations 


H 

A 


-c 

b 


C2.  It  holds  that  dflld  >  0  for  all  d  such  that  A+d  —  0  and  A0d  >  0. 

In  his  proof,  Contesse  derives  an  alternative  formulation  of  Condition  C2  involv¬ 
ing  the  set  of  generators  for  the  finite  cone 

{p  |  /l+p  =  0,  A0p  >  0}. 

This  formulation  is  described  in  Theorem  3.1  below.  For  the  sake  of  completeness, 
Contesse’s  proof  is  reviewed  with  a  notation  relevant  to  our  assumptions. 


3.  A  Proof  of  the  Optimality  Conditions 


Let  K+  denote  the  m+  x  n  matrix  whose  j-th  column  y+j  is  defined  to  be  the  unique 
vector  satisfying  the  equation 


/  11  Al  Aj  ) 

(  y+}  ^ 

(  0  > 

A+  0  0 

-Pi 

- 

e; 

V  A0  0  0  y 

\  -Vi  J 

V  0  / 

and  let  Y0  denote  the  m0  x  n  matrix  whose  j-th  column  y0J  is  defined  to  be  the 
unique  vector  such  that 


(  H  <  AJ  ) 

(  Voj  } 

{  °  ^ 

o 

o 

+ 

-A, 

— 

0 

V  Ao  0  0  j 

1  ) 

l  0  / 

3.  .4  Proof  of  the  Optimality  Conditions 
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Equations  (3.1)  and  (3.2)  imply  that  the  computation  of  y+J  and  y0J  involves 
solving  the  KKT  equations  with  a  unit  right-hand  side.  For  a  detailed  discussion  of 
the  properties  of  the  KKT  equations  in  this  context,  see  Gould  [Gou86,  Theorem 
2-3). 

Given  Y+  and  Y0,  let  M  denote  the  n  x  n  matrix  M  =  (  Z  T+  Y0  ). 

Lemma  3.1.  The  matrix  M  is  nonsingular. 

Proof.  It  is  enough  to  show  that  the  columns  of  M  are  linearly  independent.  Assume 
that 

Mv  =  Zvz  +  T+u+  +  »0  =  0. 

Successive  premultiplication  of  Mv  by  A+  and  A0  gives  n+  =  0  and  t>0  =  0.  Since 
the  columns  of  Z  are  independent,  it  follows  that  vz  =  0.  | 


Lemma  3.2.  The  sets 

{p  |  Ap  >  0}  and  {p  |  p  =  Zvz  -f  T+e+  +  Y0v 0,  v+  >  0,  v0  >  0} 
are  identical. 

Proof.  We  may  write  p  in  the  form  p  =  Zvz  +  Y+  v+  -f  Y0 v0 ,  where  v+  and  v0  are 
nonnegative  vectors.  Prernultiplication  of  p  by  A  yields 


From  Lemma  3.1  it  follows  that  the  columns  of  Z,  T+  and  V'0  span  9?n.  However,  if 
any  component  of  u+  or  r0  is  negative,  the  vector  Ap  will  have  at  least  one  negative 
component.  | 

Verification  of  the  optimality  of  x  is  now  equivalent  to  finding  a  local  solution 
of  the  quadratic  program 

minimize  gTp  +  \pTHp 

(3.3) 

subject  to  Ap  >  0. 

Lemma  3.3.  The  vector  x  is  a  local  minimizer  of  (1.1)  if  and  only  if  zero  is  a  local 
rninimizer  of  (3.3). 

Proof.  The  Taylor-series  expansion  of  gives  grp  +  ^pTHp  =  <p(z  +  p)  -  <p(z).  The 
vector  Ap  is  nonnegative  if  and  only  if  A(x  +  p)  >  Ax.  Since  every  active  constraint 
is  included  in  A,  the  point  x  will  not  be  a  local  minimizer  of  (1.1)  if  and  only  if 
there  exists  an  infinite  sequence  converging  to  z  such  that  Axk  >  b  and 

y{xk)  <  <p(x).  We  need  consider  only  those  constraints  in  the  working  set  because 
assumption  A2  guarantees  that  if  converges  to  z,  all  other  constraints  will 
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bo  satisfied  for  k  sufficiently  large  Similarly,  the  zero  vector  will  not  be  a  local 
minimizer  of  (3.3)  tf  and  only  if  there  exists  an  infinite  sequence  {/•»*■’ ;  converging 
to  zero  such  that  .4 pk  >  0  and  g,pk  +  \pkl  ll pk  <  0.  The  proof  is  complete  if  we  let 
xk  =  x+pk.  | 


Lemma  3.4.  All  dements  of  the  matrices  ZrII\\  and  Z1  HY0  air  zero. 

Proof.  Direct  substitution  in  (3.1)  yields  /f7//y4J  -  0  for  j  =  1,  . . . .  m+  arid  direct 
substitution  in  (3.2)  yields  Zl  11  yn]  —  0  for  j  —  1 . m, „.  | 


Lemmas  3.3  and  3.1  are  now  combined  to  show  that  the  verification  of  x  as  a 
local  minimizer  is  achieved  by  solving  the  QP  problem 


minimize 
subject  to 


hl'A  +  \rlZJHZvz  +  //V+e+  +  r'JV/'// V0  e0  +  WXf'in\ 

tv  >  0.  tv  >  0. 


(3.4) 


Lemma  3.5.  The.  vector  x  is  a  local  minimizer  of  (1.1 )  if  and  only  if  ze  w  is  a  local 
minimizer  of  (■}■■(). 

Proof.  P  roblem  (3.4)  is  derived  from  problem  (331)  by  using  the  transformation 

p  -  Mr  =  Zvz  +  Y+  t>+  +  y0 r0. 

Assumption  A4.  equations  (3.1)  and  (3.2),  and  Lemma  3.4  are  used  to  simplify  the 
objective  function.  The  feasible  region  is  obtained  by  using  Lemma  3.2.  Finally. 
Lemma  3.3  implies  that  "ero  is  a  local  minimizer  of  (1.1)  if  and  only  if  it  is  a  local 
minimizer  of  (3.4).  | 

Using  these  results  it  is  possible  to  pose  the  problem  of  determining  local  opti¬ 
mality  as  a  copositivitv  problem,  as  the  following  theorem  shows. 


Theorem  3.1.  (Contesse  [Con80])  The  point  x  is  a  loceil  minimizer  of  (1.1)  if 
and  only  if  Yjlf  Ya  is  copositive. 


Proof.  Assume  that  YjllY0  is  not  copositive.  T'en  there  exists  a  nonnegative 
vector  i>0  such  that  vjYj HY0v0  is  negative  and  zero  is  not  a  local  minimizer  of  (3.4). 
Lemma  3.5  implies  that  x  is  not  a  local  minimizer  of  (1.1). 

Assume  that  YjHY0  is  copositive.  If  zero  is  not  a  local  minimizer  of  (3.4),  there 
must  exist  an  infinite  sequence  {v* converging  to  zero  such  that 


PTA  +  hvz ‘z'irzvl  +  K'n'  11 Y A  +  HYA  +  A,yA^A0  <  o. 


1  ..kToT, 


I  ..kT-wT 


kTvT  i 


1  ..ArTVT, 


where  vk  and  vk  are  nonnegative.  Since  ZTH Z  is  positive  definite  and  Yjll V’0  is 
copositive,  it  must  hold  that 


IW  +  \KTyJ }I Y A  +  vkTYj It Y A  <  0. 


.{.  On  the  Copositivity  of  a  Matrix 
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At  least  one  component  of  vk  must  be  positive,  since  the  left-hand  side  is  zero  when 
(A  is  zero.  Since  /t+  is  a  positive  vector,  it  must  have  a  positive  least  component 
ftmm.  and  we  may  write 


/W Ti'+  +  HTYjHY+vll  +  vkjY? HY0vk  <  0, 


where  e  is  a  suitably  dimensioned  vector  with  unit  components.  If  both  sides  of  this 
last  equation  are  divided  by  the  positive  quantity  eTvk,  we  obtain  the  inequality 


+ 


—L_r  kTY7 

2eV  +  >  + 


HY+v*  + 


eTv 


Tv\TYj HYovk0  <  0. 


(3.5) 


If  we  now  consider  this  inequality  as  k  goes  to  infinity,  we  note  that  pmi„  must  be 
nonpositive,  which  contradicts  the  assumption  that  /r+  is  a  positive  vector.  Hence, 
the  zero  vector  is  a  local  ntinimizer  of  (3.4)  and  Lemma  3.5  implies  that  x  is  a  local 
minimizer  of  ( 1.1 ).  | 

From  this  theorem,  it  follows  that  if  we  are  able  to  check  the  m0  x  m0  matrix 
Yj II  V'0  for  copositivity,  we  are  able  to  determine  if  x  is  a  local  minimizer. 


4.  On  the  Copositivity  of  a  Matrix 

It  was  shown  in  the  previous  section  that  the  verification  of  optimality  of  a  dead 
point  x  is  equivalent  to  checking  if  the  m0  x  m0  matrix  YjlIY0  is  copositive.  Once 
}'0  is  computed,  the  matrix  Yj II Y0  may  be  calculated  by  performing  direct  matrix 
multiplications.  However,  the  following  lemma  shows  that  the  m0  solutions  of  the 
equation  (3.2)  for  j  =  1,  . . . ,  m0  provide  the  matrix  Y0TIIY0. 

Lemma  4.1.  If  0}  satisfies  (3.2),  then  YjlIY0e ,  =  9}  for  j  =  1 . m0. 

Proof.  Direct  substitution  in  (3.2)  yields  yllHy0}  =  e, I 

Copositive  matrices  have  been  studied  extensively  (see,  e.g.,  Cottle  et  al.  [CHL70] 
and  Pereira  [Per72]).  The  problem  of  deciding  if  a  given  matrix  is  copositive  has  been 
shown  to  be  NP-hard,  see  Murty  and  Kabadi  [MK87]  and  Pardalos  and  Schnitger 
[PS88].  Therefore,  no  computationally  tractable  method  for  solving  the  general 
problem  is  known. 

However,  there  are  special  situations  in  which  a  matrix  may  be  simply  checked 
for  copositivity.  Two  such  situations  are  discussed  in  the  following  lemmas. 

Lemma  4.2.  If  the  elements  of  the  matrix  Yj II Y0  are  nonnegative,  it  is  copositive. 

Proof.  If  Yj IIY0  is  not  copositive,  there  must  exist  a  nonnegative  vector  v0  such 
that  VqYJ II Yov0  <  0.  This  is  clearly  impossible  if  all  elements  of  }'0TIIY0  are  non¬ 
negative.  | 
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Lemma  4.3.  If  a  diagonal  element  of  \'J HYa,  say  yjx  H  y0l  is  negative,  the  matrix  is 
not  copositivc.  Moreover,  the  vector  y0 ,  is  a  feasible  direction  of  negative  curvature. 

Proof.  If  '//, // yoi  <  0,  then  clearly  y0l  is  a  direction  of  negative  curvature.  Lemma 
3.2  implies  that  y0l  is  a  feasible  direction,  as  required.  | 

It  is  also  straightforward  to  check  for  copositiviiy  when  V0T//V'0  is  a  2  X  2  matrix 
with  nonnegative  diagonal  elements. 

Lemma  4.4.  .4  2  X  2  real  symmetric  matrix  with  nonnegative  diagonal  elements  is 
not  copositive  if  and  only  if  its  determinant  is  negative  and  its  off-diagonal  elements 
are  negative.  Mnreove  r,  if  the  ma'rix  is  not  copositive ,  the  eigenvector  corresponding 
to  the  negative  eigenvalue  is  a  positive  vector. 

Proof.  See  Cottle  et  al.  (CIIL70,  Theorem  3.1j.  | 

As  a  consequence  of  Lemma  1.1  the  following  lemma  is  immediate. 

Lemma  4.5.  Assume  that  ) 'J ll L'0  has  nonnegative  diagonal  elements.  Moreover, 
assume  that  it  has  a  2  X  2  principal  submatrix  with  negative  determinant  and  neg¬ 
ative  aJJ-diagonal  elements.  A  feasible  direction  of  negative  curvature  for  Y’JlI  V'q  is 
givt  n  by  the  n0-vtclor  whose  nonzero  elements  are  the  components  of  the  eigenvector 
corresponding  to  the  negative  eigenvalue  of  the  2x2  principal  submatrix. 

Proof.  It  follows  from  Lemma  4.4  that  it  is  possible  to  choose  the  eigenvector 
corresponding  to  the  negative  eigenvalue  of  this  principal  2x2  submatrix  with  both 
components  nonnegative.  Hence,  this  eigenvector  extended  by  zeros  in  the  remaining 
( n0  -  2)  positions  is  a  feasible  direction  of  negative  curvature.  | 

Xow  we  propose  a  scheme  for  the  verification  of  local  optimality  based  on  the 
lemmas  above.  First,  it  is  shown  that  artificial  constraints  cause  no  additional 
difficulties. 

5.  Artificial  Constraints  in  the  Working  Set 

From  the  earlier  discussion,  it  i.->  clear  that  there  may  exist  certain  dead  points 
at.  which  the  verification  of  local  optimality  is  very  difficult.  In  this  section  we 
demonstrate  that  this  inherent  difficulty  need  not  be  exacerbated  by  the  imposition 
of  artificial  constraints. 

To  simplify  the  discussion,  it  will  be  necessary  to  distinguish  between  artificial 
and  regular  constraints.  Accordingly,  we  partition  A0  and  V0  such  that 


(M 

and  Y0  =  ( 

Yh  ya  ) 

V  i,  ) 

V 

where  tire  subscript  "l<"  denotes  regular  constraints  and  the  subscript  “T'  denotes 
artificial  constraints.  Let  rnH  denote  the  number  of  rows  of  AH  and  let  mA  denote 
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the  number  of  rows  of  Aa.  Also  let  yRj  denote  the  j-th  column  of  YR  and  let  yAJ 
denote  the  j-th  column  of  YA.  When  artificial  constraints  are  present,  the  definition 
of  a  feasible  direction  will  be  changed  as  follows: 

A  vector  p  is  said  to  be  a  feasible  direction  if  A4p  >  0  and  AHp  >  0. 

Note  that  the  sign  of  the  vector  A  Ap  is  not  restricted. 

It  is  also  necessary  to  use  a  slightly  modified  version  of  assumption  4: 


A4'.  The  point  x  satisfies  the  first-order  necessary  conditions  for  optimality,  i.e., 
there  exists  a  Lagrange  multiplier  vector  p  —  ^  p+  p A  pR  ^  7,  with  /z+  >  0, 
pR  >  0  and  pA  =  0,  such  that  x  and  p  satisfy  the  KKT  equations 


(  I!  \T  Ar 

1 1  .  i  +  n  R 

,i+  o  o 

0  0 

V  A  A  o  0 


o 
0 

0  ) 


(  X  \ 

-ic 

~Ar 

\  ) 


/  -c  \ 

K 

l)R 
bA 


The  difference  between  assumptions  A4  and  A4'  is  that  the  Lagrange  multipliers 
of  the  artificial  constraints  are  required  to  be  zero.  If  an  artificial  constraint  has 
a  nonzero  multiplier,  it  could  be  deleted  from  the  working  set  to  yield  a  feasible 
descent  direction.  Therefore,  assumption  A4'  is  appropriate  for  x  being  a  constrained 
stationary  point.  Consequently,  a  point  x  is  said  to  be  a  dead  point  if  it  satisfies 
assumptions  A4'  and  A5. 

Unfortunately,  additional  dead  points  may  be  added  to  the  problem  by  imposing 
artificial  constraints.  Consider  the  problem 

minimize  -x\x2 

r£3?  2 

subject  to  -1  <  Xi  <  1  (5-1) 

-1  <  x2  <  1. 

If  the  starting  point  is  the  origin,  no  regular  constraints  are  active  and  artificial 
constraints  are  needed  to  obtain  a  positive-definite  reduced  Hessian.  If  artificial 
bound  con  ints  =  0  and  x2  =  0  are  imposed,  assumptions  A  V3,  A4'  and 
A5  are  ■  d.  However,  as  in  problem  (2.2),  the  origin  is  not  a  local  mbumizer 
and  no  f  .^.z.e  direction  of  negative  curvature  may  be  obtained  by  deleting  only 
one  arfmeia!  !->r  id. 

It  r  'ght  if  .m  necessary  that  an  arbitrary  (unknown)  number  of  artificial  con¬ 
straints  ..lust  be  deleted  to  give  a  feasible  direction  of  negative  curvature  (if  one 
“xists).  However,  we  shall  show  below  that  such  a  direction  may  be  computed  by 
making  only  one  or  two  artificial  constraints  leave  the  working  set. 

In  order  to  show  this,  we  determine  constraints  in  the  working  set  that  may  be 
deleted  yielding  a  positive-definite  reduced  Hessian.  This  may  be  determined  from 
the  solution  of  (3.1)  and  (3.2),  as  observed  by  Gould  [Gou86,  Theorem  2.3]  and 
reviewed  in  the  following  lemma. 


: 
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Lemma  5.1.  If  a  constraint  corresponding  to  a  positive  diagonal  element  of  YfilIY0 
is  deleted  from  .4,  the  resulting  reduced  Hessian  remains  positive  definite. 

Proof.  Let  y0l  correspond  to  the  deleted  constraint  ajx  >  ffi .  Lemma  3.1  implies 
that  yox  is  independent  of  the  columns  of  Z  and  it  follows  from  (3.2)  that  Ay0 ,  =  et. 
Therefore,  a  basis  for  the  new  nullspace  is  obtained  by  adding  the  column  yot  to 
Z.  Lemma  3.1  implies  that  ZTIly0l  is  zero.  Hence,  the  fact  that  yJ,Hy0x  is  positive 
implies  that  the  new  reduced  Hessian  remains  positive  definite.  | 


In  order  to  distinguish  between  artificial  and  regular  constraints  we  partition 
) fill Y0  such  that 


y£iiYr  yIhya  \ 
yJhyr  yJhya  ) 


It  follows  from  Lemma  1.3  that  if  a  diagonal  element  of  YjllYA  is  negative, 
a  feasible  direction  of  negative  curvature  can  be  computed.  By  Lemma  5.1  it  fol¬ 
lows  that  if  a  diagonal  element  of  Yj HYa  is  positive,  the  corresponding  artificial 
constraint  can  be  deleted  and  the  new  reduced  Hessian  will  be  positive  definite. 
Clearly,  unless  all  diagonal  elements  of  YAHYA  are  zero,  either  a  feasible  direction 
of  negative  curvature  can  be  computed  or  an  artificial  constraint  can  be  deleted. 


Lemma  5.2.  If  two  diagonal  elements  of  Yj IIYA,  say  yTAxIIyAl  and  yTAJ H yAJ  arc 
zero,  and  y\jl yA}  is  nonzero,  the  point  x  is  not  a  local  minimizer.  Moreover,  either 
yAi  -  yA]  or  yAl  +  yAJ  is  a  feasible  direction  of  negative  curvature. 


Proof.  Direct  calculation  yields  (yAX  +  yA  )TII(yAX  +  yAJ)  =  ~(yA,  ~  yAj)TH(yAX  - 
yAj )  =  2 yTAllIyAJ  o.  Hence,  either  yA,  +  yAJ  or  yA,  -  yA]  is  a  direction  of  negative 
curvature.  Feasibility  follows  from  the  relations  4+( yAX  ±  yAj)  =  0  and  AR(yAX  ± 

yAj)  =  o.  i 


This  lemma  demonstrates  that  unless  the  matrix  Yj HYa  is  zero,  either  a  feasible 
direction  of  negative  curvature  can  be  computed  or  an  artificial  constraint  can  be 
deleted. 


Lemma  5.3.  If  the  diagonals  ofYj IIYA  are  zero,  and  an  element  of  YR  II YA  (say 
ylRl IIyAJ)  is  nonzero,  the  point  x  is  not  a  local  minimizer  and  a  feasible  direction  of 
negative  curvature  may  be  computed. 

Proof.  Let  p  be  a  vector  of  the  form  a,yHx  +  a}yAJ.  Direct  calculation  yields  that 
p  is  feasible  if  a,  is  nonnegative.  The  quantity  pTIIp  may  be  expressed  as 

ylJhAJ  W  <*i  \ 

11  yA}  o  /  \  qj  / 

Consider  the  2x2  matrix  T  given  by 

vlJhm  yl,ffyA]  \ 
yTRxHyA}  o  ) 


pTIIp  =  (  a,  atj  )  f 
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Since  yR,IlyAJ  is  nonzero,  T  has  one  negative  and  one  positive  eigenvalue.  It  has 
or'hogona!  eigenvectors,  since  it  is  a  real  symmetric  matrix.  Hence,  a,  and  aj  may 
be  chosen  so  that  p  is  the  eigenvector  corresponding  to  the  negative  eigen\alue, 
with  a,  nonnegative.  For  those  values  of  o,  and  Oj,  the  vector  p  will  be  a  feasible 
direction  of  negative  curvature.  | 

Clearly,  whenever  a  component  of  YjHYA  is  nonzero,  either  an  artificial  con¬ 
straint  can  be  deleted  or  a  feasible  direction  of  negative  curvature  can  be  computed. 
To  summarize,  the  following  result  holds  when  artificial  constraints  are  present  in 
the  working  set. 

Theorem  5.1.  lfYjllYA  has  nonpositive  diagonal  elements,  then  x  is  a  local  min- 
irnizer  of  (1.1)  if  and  only  ifYR  I1YR  is  copositive  and  Y,T H  YA  and  Yj  11 Y A  are  zero. 

Proof.  If  YR  11 Y A  or  Yj II YA  are  nonzero,  there  exists  a  feasible  direction  of  negative 
curvature  and  x  cannot  be  a  local  minimizer. 

Assume  that  Yjt  H ) 'R  is  not  copositive.  In  this  case,  a  feasible  direction  of 
negative  curvature  may  be  computed  and  the  local  optimality  of  x  is  contradicted. 

Assume  that  Yj HYr  is  copositive  and  YRHYA  and  Yj HYA  are  zero.  Using  a 
similar  analysis  to  that  for  the  regular-constraint  case,  we  can  make  the  following 
assertions.  As  in  Lemma  3.2,  partition  the  vector  v0  such  that 


v0  = 


and  replace  the  constraint  v0  >  0  in  (3.4)  by  vR  >0.  If  x  is  not  a  local  minimizer  of 
(1.1)  there  must  exist  an  infinite  sequence  converging  to  zero  such  that 


-I - - — vkTYT HY  vk  +  — —vkTYT II Y  vk  A _ ~vkTYT HY  vk  <  0 

'2erufc  +  1  +  11  1  + 1 +  +  eTnk  V+  1  +  11  1  RVR  +  eTx,k  V+  1  +  n  1  AV A  u 


e't)1 


e'v? 


Again,  it  we  consider  this  inequality  as  k  goes  to  infinity,  we  obtain  the  required 
contradiction.  | 


Consequently,  if  assumptions  A1-A3,  A4'  and  A5  hold,  the  artificial  constraints 
will  cause  no  extra  problem  in  determining  if  x  is  a  local  minimizer.  There  remains 
the  hard  question  of  verifying  that  the  matrix  YRHYR  is  copositive. 


6.  Computation  of  Directions  of  Negative  Curvature 

In  this  section,  we  propose  an  extension  to  ICQP  methods  that  will  allow  progress  to 
be  made  at  a  dead  point.  Algorithm  6.1  provides  a  means  of  computing  a  direction 
of  negative  curvature  by  making  one  or  two  active  constraints  inactive.  Lemma  6.1 
below  indicates  that  the  algorithm  will  terminate  with  either  a  direction  of  negative 
curvature  or  the  conclusion  that  a;  is  a  local  minimizer. 


H 
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Lemma  6.1.  Algorithm  6.1  will  terminate  in  at  most  m0  steps.  Moreover,  if  ter¬ 
mination  occurs  without  the  computation  of  a  direction  of  negative  curvature ,  x  is 
a  local  rnmimizer  of  (1.1). 

Proof.  At  each  step,  either  the  algorithm  terminates  or  a  constraint  is  deleted  from 
the  working  set.  Since  there  are  only  m0  constraints  to  delete,  the  algorithm  must 
stop  in  at  most  m0  steps. 

If  Yj HYa  has  a  positive  diagonal  element,  the  corresponding  artificial  constraint 
is  deleted.  Since  this  deletion  will  be  repeated  until  every  diagonal  element  of 
yJhya  is  nonpositive,  we  may  assume  that  YjHYA  has  nonpositive  diagonal  ele¬ 
ments.  At  this  point,  if  no  direction  of  negative  curvature  is  computed,  the  matrices 
YjHYA  and  YjHYA  will  be  zero  at  each  subsequent  step  of  the  algorithm.  Either 
the  algorithm  detects  that  the  matrix  YR HYr  is  copositive,  or  a  constraint  cor¬ 
responding  to  a  positive  diagional  element  of  YR  HYR  is  deleted.  If  the  algorithm 
terminates  without  having  computed  a  direction  of  negative  curvature,  the  algo¬ 
rithm  has  determined  that  a  local  minimizer  has  been  found  with  respect  to  the 
constraints  that  are  still  present  in  AR.  However,  this  conclusion  still  holds  if  the 
deleted  constraints  are  added  again,  since  deletion  of  constraints  may  only  increase 
the  size  of  the  feasible  region.  1 

Hence,  if  Algorithm  6.1  does  not  terminate  at  a  given  step,  a  constraint  with  a 
positive  diagonal  element  of  Y0THYn  is  deleted.  Recall  that  Lemma  5.1  implies  that 
the  new  reduced  Hessian  is  positive  definite  whenever  a  constraint  corresponding  to 
a  positive  element  of  Y0rHYQ  is  deleted. 

The  amount  of  work  needed  at  each  step  may  be  reduced  by  updating  Y0  and 
YjH Ya .  To  show  this,  we  assume  that  the  normal  of  the  constraint  ajx  >  /?,■  is 
deleted  from  A0  corresponding  to  a  positive  diagonal  element  of  Y0TIIY0.  Partition 
A0  such  that 


In  order  to  state  the  results  in  compact  form,  let  A  denote  the  matrix  whose  j-th 
column  is  A,  in  (3.2)  and  let  0  denote  the  matrix  whose  ji-th  column  equals  6r 
With  this  partition  of  A0,  let  the  induced  partition  of  Y0,  A  and  0  be  given  by 

Yo  =  (  Yoi  y0,  )  ,  A  =  (  Ai  A,  )  and  0  =  (  &XX  6'X  j  . 

'  \  01  i  v i,  J 

With  this  partition  equation  (3.2)  may  be  written  in  compact  form  as 
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Algorithm  6.1.  .4 rj  algorithm  for  finding  a  direction  of  negative  curvutun 
repeat 

Compute  y;T//y;;  Initialize  mA  and  mR\ 
if  (mA  >  0)  then 

k  argument  satisfying  max,  y\xHyAl\ 
if  {yAkUVAk  >  °)  then 

Delete  artificial  constraint  k\  go  to  again;  (see  Lemma  5.1) 

end  if; 

k  —  argument  satisfying  min,  yAi H yAl; 
if  {</AkHyAk  <  0)  then 

p  —  ijai<\  Oo  to  exit;  (see  Lemma  4.3) 

end  if; 

kj  <—  arguments  satisfying  maxXJ  |yJ,//yAJ|; 

if  (ylkH  'JAi  f  °)  then 

P  —  y Ak  i  yAi\  go  to  exit;  (see  Lemma  5.2) 
else  if  ( mH  >  0)  then 

kj  —  arguments  satisfying  maxltJ  \yAXDyR]\; 

if  (yTAknyni  ±  °)  then 

Compute  p\  go  to  exit;  (see  Lemma  5.3) 
end  if; 
end  if; 
end  if; 

if  (mR  =  0)  or  {minXJ  yTRlHyR]  >  0)  then 

x  is  a  local  minimizer;  go  to  exit;  (see  Lemma  4.2) 

end  if; 

k  «—  argument  satisfying  minx  yRt II yRl ; 

if  (yTRkuyRk<Q)  then 

V  Vnk\  go  to  exit;  (see  Lemma  4.3) 

end  if; 

for  i  *—  1  to  mR  do 

for  j  «—  i  -f  1  to  mR  do 

negdet.  -  {yTRxHyRl  yTRjHyR]  -  yTRxHyRj2  <  0); 
if  ( negdet )  and  {y^xHyR]  <  0)  then 

Compute  p\  go  to  exit;  (see  Lemma  4.5) 
else  if  (mR  =  2)  then 

x  is  a  local  minimizer;  go  to  exit;  (see  Lemma  4.4) 

end  if; 
end  do; 
end  do; 

k  <—  argument  satisfying  max;  yJ,//yR;; 

Delete  regular  constraint  k\  (see  Lemma  5.1) 
label  again: 
until  exit  occurs; 
label  exit: 
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Let  Y0,  A  and  0  denote  the  solution  of  (3.2)  in  the  next  step  of  Algorithm  6.1. 
Then,  Y0,  A  and  0  will  satisfy  the  equation 
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Lemma  6.2. 
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fhe  quantities  0  and  Y0  may  be  obtained  from  the  solution  of  (6.1) 


0-0  ii- 


o,, 


and  Y0  -  Y0i  - 
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Proof.  The  matrices  0  and  Y0  satisfy  the  equation 


(  II  A\  Al \  ax  \ 

f  M 

(  0  ^ 

A+  0  0  0  1 

-A 

0 

O 

O 

O 

o 

-0 

/ 

o 

o 

o 

l  0  ) 

l  affo  / 

(6.2) 


Equations  (6.1)  and  (6.2)  imply  that  the  barred  quantities  may  be  obtained  from 
the  equations 


n  -  n,  t  y0iai  Y0 

(6.3a) 

A  =  A,  +  A  ,ajY0 

(6.3b) 

0  =  0U  +  9ixajY0 

(6.3c) 

o  =  0lt  +  sltafro. 

(6.3d) 

It  follows  from  Lemma  4.1  that  0  =  YjHY0.  Hence,  0  is  a  symmetric  matrix 
with  0,,  =  6jx.  Equation  (6.3d)  implies  that  0jx  -f  6xtajYa  =  0.  The  fact  that 
ajx  >  fit  is  associated  with  a  positive  diagonal  element  of  YjNY0  implies  that  is 
positive.  Substitution  in  (6.3a)  and  (6.3c)  yields  the  desired  result.  | 


Hence,  only  a  rank-one  modification  of  Y0  and  Y^HY0  is  needed  at  each  step  of 
Algorithm  6.1. 


Lemma  6.3.  Assume  that  yJtHy0j  is  zero  and  yJiHy0i  is  positive  at  one  step  of 
Algorithm  6.1.  Also  assume  that  the  constraint  with  normal  A^e;  is  deleted  at  this 
step.  At  the  next  step,  the  column  of  YjHY0  corresponding  to  the  constraint  with 
normal  Aje}  is  modified  only  by  deletion  of  the  zero  element  yjt  H  y0J . 

Proof.  Lemma  6.2  implies  that  the  rank-one  modification  of  column  j  is  zero  when 
yJ,Hy0}  is  zero.  | 

Lemma  6.4.  If,  in  one  step  of  Algorithm  6.1,  it  holds  that  the  matrices  YjHYA 
andYjHY,  are  zero ,  then  they  will  remain  zero. 


7.  Changes  in  the  Working  Set 
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Proof.  Since  both  the  matrices  YRHYA  and  Yj HYa  are  zero,  the  only  way  the 
algorithm  does  not  terminate  is  when  a  regular  constraint  corresponding  to  a  posi¬ 
tive  diagonal  element  of  Yj HYr  is  deleted.  Lemma  6.3  implies  that  the  matrices 
Yj II VA  and  Yj HYa  will  remain  zero.  Only  one  column  of  zeros  from  Yj HYa  is 
deleted  at  each  step.  | 

Hence,  once  the  matrices  V'/ H YA  and  YAIIYA  are  zero,  they  remain  zero. 

Lemma  6.5.  If  YR HYr  is  positive  semide finite  and  YR HYa  and  YjlIYA  are  zero, 
then  Algorithn,  6.1  will  resolve  that  x  is  a  local  minimizer  in  at  most  mR  steps. 


Proof.  Lemma  6.4  implies  that  the  matrices  YR  HYA  and  Yj  HYA  will  remain  zero 
until  the  algorithm  terminates.  Hence,  the  only  iteration  when  the  algorithm  does 
not  halt  is  when  regular  constraints  corresponding  to  positive  diagonal  elements  of 
Y'J HYr  are  deleted.  Therefore,  at  most  mR  steps  may  be  taken  in  the  algorithm. 

Assume  that  the  algorithm  terminates  without  determining  that  x  is  a  local  min¬ 
imizer.  It  fellows  that  a  direction  of  negative  curvature  must  have  been  computed. 
But  Lemma  6.2  implies  that  the  matrix  0  of  the  next  step  is  obtained  as 


0  =  0n 


0u  ' 


Sylvester’s  law  of  inertia  implies  that  ln(0)  =  ln{0 )  -  In(0„).  At  the  initial 
iteration,  0  is  positive  semidefinite.  The  value  of  the  scalar  is  positive.  Hence, 
0  will  have  no  negative  eigenvalues.  It  follows  by  induction  that  no  direction  of 
negative  curvature  can  be  computed.  | 


Hence,  if  YjllYR  is  positive  semidefinite,  Algorithm  6.1  determines  that  x  is  a 
local  minimizer. 


7.  Changes  in  the  Working  Set 

In  this  section  the  changes  in  the  working  set  are  described.  In  the  proposed  algo¬ 
rithm,  either  one  or  two  constraints  in  A  will  become  inactive.  In  an  ICQP  method, 
only  one  constraint  is  added  or  deleted  at  a  time.  However,  we  shall  give  a  scheme 
that  allows  deletion  of  two  constraints  at  a  dead  point,  maintaining  the  properties 
of  an  ICQP  method,  i.e.,  the  reduced  Hessian  having  at  most  none  nonpositive 
eigenvalue  and  the  working-set  matrix  having  full  row  rank. 

When  a  direction  of  negative  curvature,  p,  is  computed,  the  objective  function 
is  strictly  decreasing  along  that  direction.  The  boundedness  of  <p  in  the  feasible 
region  guarantees  that  a  sufficiently  large  step  along  p  will  violate  a  constraint.  Let 
Ok  denote  the  normal  of  the  first  constraint  that  is  violated.  In  order  to  determine 
how  to  update  A,  it  is  necessary  to  know  if  a*  is  dependent  on  the  rows  of  A.  The 
following  lemma,  given  by  Gill  et  al.  [GMSW88],  shows  how  linear  independence 
may  be  checked. 
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Local  Minimizers  in  ICQPs 


Lemma  7.1.  Consider  the  equations 


H 

A 


(7.1) 


The  vector  a*  is  dependent  on  the  rows  oj  A  if  and  only  if  the  vector  u  is  zero  in 
the  solution  of  (7.1). 


Proof.  Suppose  that  a k  is  dependent  on  the  rows  of  A.  In  this  case,  there  must 
exist  a  vector  v  such  that  a*.  =  ATv,  and  u>  is  zero  in  the  solution  of  (7.1). 

Assume  that  u  is  zero  in  the  solution  of  (7.1).  It  follows  that  ak  =  ATv,  and  a*,, 
is  dependent  on  the  rows  of  A.  | 


When  the  algorithm  is  applied,  either  one  or  two  constraints  leave  the  working 
set.  The  following  sections  show  how  to  update  the  working-set  matrix. 

7.1.  One  constraint  becomes  inactive 

Assume  that  p  is  given  by  p  =  y0l  and  let  ajx  >  (5t  denote  the  constraint  that  leaves 
the  working  set. 

Lemma  7.2.  Assume  that  p  is  computed  by  deleting  one  constraint  from  the  work¬ 
ing  set.  If  ak  is  independent  of  the  rows  of  A,  it  is  added  to  A,  white  a,  is  maintained 
in  A  as  an  artificial  constraint.  If  ak  is  dependent  on  the  rows  of  A,  ak  and  a,  are 
exchanged.  In  either  case,  the  resulting  reduced  Hessian  is  positive  definite  and 
working-set  matrix  has  full  row  rank. 

Proof.  If  at  is  independent  of  the  rows  of  A,  the  new  reduced  Hessian  remains 
positive  definite  since  only  one  more  constraint  is  added  to  the  working  set.  Also, 
the  new  working-set  matrix  has  full  row  rank. 

Now  assume  that  ak  is  dependent  on  the  rows  of  A.  If  ak  and  a,  are  exchanged, 
the  rows  of  the  new  working  set  will  span  the  same  space  as  the  rows  of  A.  Hence , 
the  new  reduced  Hessian  is  positive  definite.  Also,  the  new  working-set  matrix  has 
the  same  number  of  rows  as  the  old  one,  and  therefore  it  has  full  row  rank.  | 


Hence,  after  having  either  added  ak  or  exchanged  a*  and  a,,  the  new  reduced 
Hessian  is  positive  definite  and  the  new  working-set  matrix  has  full  row  rank. 

7.2.  Two  constraints  become  inactive 

Assume  that  p  is  given  by  p  =  otiy0i  +  ajy0j,  where  a;  and  aj  are  both  nonzero.  Let 
a,  and  a3  denote  the  normals  of  the  constraints  which  leave  the  working  set. 

Lemma  7.3.  Assume  that  ak  =  A2v2  +  a^v,  +  ajV}.  Then  it  cannot  hold  that 

v ,  =  Vj  =  0. 


S.  Verification  of  Local  Optimality 


19 


Proof.  Assume  that  a *  =  Ajv2-  Premultiplication  by  pT  yields  pTa *  =  0.  But  this 
could  not  hold  since  a k  becomes  violated  when  a  sufficiently  large  step  along  p  is 
t  aken .  | 

Lemma  7.4.  Assume  that  p  is  computed  by  making  two  active  constraints  inactive. 
If  a;,  is  independent  of  the  rows  of  A,  it  is  added  to  A,  while  a,  and  a}  are  maintained 
as  artificial  constraints.  If  is  dependent  on  the  rows  of  A  and  1 v, |  >  \vj\,  a *  and 
a,  are  exchanged.  If  a^  is  dependent  on  the  rows  of  A  and  |n,J  <  | ^  | ,  a*,  and  a}  are 
exchanged.  In  each  case,  the  new  reduced  Hessian  is  positive  definite  and  the  new 
working-set  matrix  has  full  row  rank. 

Proof.  Assume  that  a k  is  independent  of  the  rows  of  A.  The  new  reduced  Hessian 
remains  positive  definite  since  only  one  more  constraint  is  added  to  the  working  set. 
Also,  the  new  working-set  matrix  has  full  row  rank. 

Assume  that  a k  is  dependent  on  the  rows  of  A.  Lemma  7.3  implies  that  at  least 
one  of  the  scalars  w,  and  Vj  is  nonzero.  Hence,  by  performing  the  specified  exchange, 
the  rows  of  the  new  working  set  will  span  the  same  space  as  the  rows  of  A.  Hence, 
the  new  reduced  Hessian  will  be  positive  definite.  Also,  the  new  working-set  matrix 
has  the  same  number  of  rows  as  the  old  one,  and  therefore  it  has  full  row  rank.  | 

Hence,  after  having  either  added  or  exchanged  either  a k  and  a,  or  and  a., 
the  new  reduced  Hessian  is  positive  definite  and  the  working-set  matrix  has  full  row 
rank. 

8.  Verification  of  Local  Optimality 

In  this  section  we  describe  a  complete  algorithm  for  checking  if  a  given  dead  point  x 
is  a  local  minimizer.  In  Algorithm  8.1,  a  direction  of  negative  curvature  is  found  by 
making  one  or  two  constraints  leave  the  working  set  at  a  time.  If  no  such  direction 
exists,  Algorithm  6.1  yields  the  result  that  x  is  a  local  minimizer. 

If  constraints  corresponding  to  positive  diagonal  elements  of  Yj HYR  are  deleted 
in  Algorithm  6.1,  assumption  A2  will  no  longer  hold.  In  this  case,  if  a  direction  of 
negative  curvature  is  computed,  the  resulting  maximum  feasible  step  could  be  zero 
and  there  is  a  danger  of  cycling. 

However,  if  Algorithm  8.1  terminates,  it  will  provide  either  a  feasible  direction 
of  negative  curvature  along  which  a  nonzero  step  may  be  taken  or  the  information 
that  x  is  a  local  minimizer.  As  shown  in  Section  6,  the  algorithm  will  terminate 
with  the  information  that  x  is  a  local  minimizer  in  the  special  case  when  the  matrix 
Y£ HYr  is  positive  semidefinite. 

9.  Conclusions 

When  solving  a  general  quadratic  programming  problem  there  may  exist  certain 
dead  points  at  which  it  is  very  difficult  to  verify  optimality.  We  emphasize  that  this 
difficulty  is  inherent  to  the  problem,  and  is  independent  of  the  solution  method. 


Local  Minimize.rs  in  ICQPs 


Algorithm  8.1.  An  algorithm  checking  for  local  optimality 

repeat 

Apply  Algorithm  6.1; 

local .minimizer «—  Algorithm  6.1  implies  that  x  is  a  local  minimizer; 
if  (local .minimi:cr)  then 
go  to  exit; 

else 

a*  >—  constraint  that  is  first  violated  along  p; 
qf  —  maximum  feasible  step  along  p; 

Soivc(!!  'o  )(“)=(  o );  ,sco'-'mn’ari1 

indep  —  ( | ] uz  1 1  >  0); 

nr.inactiv  *—  number  of  constraints  that  become  inactive; 
if  (nr.  inactiv  =  1)  then 

«,  —  constraint  that  becomes  inactive; 
if  (indep)  then 
Add 

else 

Exchange  a*  and  «,•;  (see  Lemma  7.2) 

end  if; 

else 

n,,«j  —  constraints  that  become  inactive; 
if  (indep)  then 
Add  at; 

else 

if  ( | •/.»,- 1  >  ji’jl)  then 

Exchange  a *  and  a,;  (see  Lemma  7.4) 

else 

Exchange  a*  and  a}\ 

end  if; 
end  if; 
end  if; 

if  (oF  >  0)  then 
x  *—  x  +  aFp\ 
local  -minimizer  «—  false; 
go  to  exit; 

end  if; 
end  if; 

until  too  many  iterations; 
label  exit; 


9.  Conclusions 
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In  this  paper,  the  verification  of  optimality  has  been  discussed  within  the  con¬ 
text  of  an  inertia-controlling  method.  We  have  derived  a  computational  method 
appropriate  for  general  ICQP  methods,  that  will  attempt  to  determine  if  a  dead 
point  is  a  local  minimizer.  The  use  of  artificial  constraints  may  introduce  additional 
dead  points.  It  has  been  shown  that  the  new  procedure  does  not  terminate  at  such 
points,  unless  they  are  local  minimizers. 

However,  the  verification  of  optimality  in  the  general  case  is  an  NP-hard  problem, 
so  we  would  not  expect  to  find  a  procedure  capable  of  solving  a  general  problem 
in  a  reasonable  amount  of  computational  effort.  In  our  scheme,  there  is  a  potential 
danger  of  cycling,  and  techniques  similar  to  anti-cycling  procedures  used  in  linear 
programming  are  needed  to  resolve  this  problem. 
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